Papers by Yuki M Asano
Bitune: Leveraging Bidirectional Attention to Improve Decoder-Only LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Decoder-only large language models rely on masked causal attention, which limits expressiveness by restricting information flow to one direction. |
| Approach: | They propose a method that incorporates bidirectional attention into prompt processing to enhance pretrained decoder-only LLMs. |
| Outcome: | The proposed method shows significant improvements in commonsense reasoning, arithmetic, and language understanding tasks. |